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Program description^ 



Ladders to Literacy is a supplemental early literacy curriculum 
published in Ladders to Literacy: A Kindergarten Activity Book. The 
program targets children at different levels and from diverse cultural 
backgrounds— those who are typically developing, have disabilities, 
or are at risk of reading failure. The activities are organized into 



three sections with about 20 activities each: print awareness, pho- 
nological awareness skills, and oral language skills. While a Ladders 
to Literacy curriculum is also available for preschool students {Lad- 
ders to Literacy: A Preschooi Activity Book), this intervention report 
focuses on the Kindergarten version of the curriculum. 



Research 



Four studies of Ladders to Literacy met the What Works Clearing- 
house (WWC) evidence standards with reservations. The studies 
included 760 students from Kindergarten classrooms at more than 
14 elementary schools in urban and rural Midwest districts.^ The 



WWC considers the extent of evidence for Ladders to Literacy to 
be moderate to large for alphabetics and comprehension and small 
for fluency. No studies that met WWC evidence standards with or 
without reservations addressed general reading achievement. 



Effectiveness 



The Ladders to Literacy program was found to have potentially positive effects on alphabetics and fluency and mixed effects on 



comprehension. 



Rating of effectiveness 



Alphabetics 


Fluency 


Comprehension 


General reading 
achievement 


Potentially positive 


Potentially positive 


Mixed effects 


na 


effects 


effects 







Improvement index^ 



Average: +25 percentile +26 percentile points 
points 

Range: +8 to +47 
percentile points 



Average:+9 percentile na 
points 

Range:+1 to +17 
percentile points 



na = not applicable 

1. The descriptive information for this program was obtained from publicly available sources: the program’s and distributor’s web sites f www.wri-edu.org/ 
ladders : wu i/w.brookesDubllshi^-iQ com : downloaded April 2007) and the research literature (O’Connor, 1999; Fuchs, Fuchs, Thompson, Al Otaiba, Yen, 
Yang et al., 2001). The WWC requests developers to review the program description sections for accuracy from their perspective. Further verification of 
the accuracy of the descriptive information for this program is beyond the scope of this review. 

2. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

3. These numbers show the average and range of improvement indices for three of the four studies. Cne additional study that showed positive effects in 
alphabetics was not included in this average and range because effect sizes were not calculated at the student level. 
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Developer and contact 

Ladders to Literacy was developed as a preschool to Kindergar- 
ten curriculum by Drs. Notarl-Syverson and O’Connor in 1993. 
The kindergarten Ladders to Literacy book by Drs. O’Connor and 
Notari-Sy verson was first published in 1998 and a second edition 
was published in 2005. The Ladders to Literacy books are dis- 
tributed by Brookes Publishing Company. Address: Brookes Pub- 
lishing Co., PC. Box 10624, Baltimore, MD 21285-0624. Email: 
custserv@brookespublishina.com . Web: www.brookespublishina. 
com . Telephone: (800) 638-3775. For professional development 
training, see http://www.brookespublishina.com/onlocation . 

Scope of use 

According to the developers, the Ladders to Literacy activities 
and professional development have been field-tested in a variety 
of kindergarten settings. These include sites that serve young 
children with disabilities in inclusive and special education 
settings, as well as sites with small and large groups of children 
reflecting a range of interests and abilities. 

Teaching 

This supplemental curriculum is published in the book Ladders 
to Literacy: A Kindergarten Activity Book, which focuses on 
developing early language and literacy skills. Ladders to Literacy: 
A Kindergarten Activity Book addresses three components: print 
awareness, phonological awareness, and oral language skills. 
Typically the teacher chooses activities that can be included in 
the current classroom routines and that require little preparation 
time. There are approximately 20 activities in each of the three 
sections. Some activities can be included weekly, monthly, or 
on a one-time basis. Most of the activities are designed for large 
groups and follow a similar format. 



Print knowledge activities are designed to help children 
construct meaning by answering the teacher’s questions about 
environmental print, conventions of print, letter names, and letter 
sounds. In addition to books, teachers use newspapers, menus, 
recipes, signs, messages, and writing experiences as learning 
tools. For the phonological awareness activities, children identify 
environmental sounds, repeat words or phrases in songs and 
nursery rhymes, manipulate phonemes in games of alliteration, 
and blend and segment words. The oral language section con- 
tains a collection of activities that provide conversational topics 
intended to facilitate language development. Using storybooks, 
pictures, objects, and films, teachers lead the children in labeling 
and describing objects or events by using questions intended to 
expand on the child’s utterances. Activities in one section (oral 
language) reference activities in other sections to encourage 
teachers to integrate activities across skills. 

The teachers use the program’s observational checklist to 
determine what tasks and teaching strategies in the lesson are 
developmentally appropriate for each child. With this information, 
teachers scaffold instruction by applying more open-ended 
questioning and feedback to children who need less support and 
more explicit instruction to children in need of more support. 

Cost 

The Ladders to Literacy: A Kindergarten Activity Book costs 
$49.95. The manual contains all the components of the program, 
including its theoretical framework, lesson plans, assessment 
checklists, and home activities for parents and children. Profes- 
sional development for Ladders to Literacy is available for an 
extra cost and consists of a one- or two-day on-site seminar on 
how to use the curriculum. 
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Six studies reviewed by the WWC investigated the effects of 
Ladders to Literacy. Three studies (O’Connor, 1999, Study A: 
Intensive Professional Development; O’Connor, 1999, Study 
B; Traditional Professional Development; and O’Connor et al., 
1996) were quasi-experimental designs that met WWC evidence 
standards with reservations. One study (Fuchs et al., 2001) was 
a randomized controlled trial with randomization problems that 
met the WWC standards with reservations."^ The remaining two 
studies did not meet WWC evidence screens. 

Met evidence standards with reservations 

O’Connor (1999, Study A: Intensive Professional Development) 
examined outcomes of Kindergarten students in a large urban 
school district. Students in the intervention and comparison 
groups received the same district-sponsored pre-reading cur- 
riculum. Students in the intervention group also did Ladders to 
Literacy activities. Students in Kindergarten were identified, pre- 
tested, matched, and divided into two groups. The WWC review 
of this study focused on the comparison of 64 typical learners 
in the intervention group with 41 students in the comparison 
group.® 



O’Connor (1999, Study B: Traditional Professional Develop- 
ment) examined outcomes of Kindergarten students in a large 
Midwestern rural school district. Seventeen teachers were 
assigned to Ladders to Literacy or the comparison condition. 
Students in the intervention and comparison groups received the 
same district-sponsored pre-reading curriculum, but students in 
the intervention group also used Ladders to Literacy activities. In 
the analysis of the full sample,® 192 students from nine classrooms 
were in the intervention group and 126 students from eight class- 
rooms were in the comparison group. In this larger replication of 
Study A, teachers received less intensive professional training. 

O’Connor et al. (1996) examined effects of Ladders to Literacy 
on Kindergarten students in a large urban school district who 
were instructed by five teachers.^ Two transition teachers were 
randomly assigned to intervention or comparison conditions. The 
three regular classroom teachers were not randomly assigned to 
the treatment or comparison condition. Students were matched 
by type of classroom (general or repeating kindergarteners). The 
WWC focused on the portion of the sample that included 42 
students from three classrooms in the intervention group and 24 
students from two classrooms in the comparison group. 



4. The study was designed as a stratified randomized controiied triai. As students were seiected for inciusion, names were offered to teachers for their 
review and adjustment. At that point, randomization was not maintained. The WWC examined pretest scores to ensure that intervention and comparison 
groups were comparable, so the study met WWC evidence standards with reservations. 

5. The intervention and comparison groups were divided by ability level (at-risk learners and typical learners). The at-risk subgroup was also examined in 
the study but the groups were not equivalent at pretest according to WWC analysis and therefore did not meet standards. 

6. The sample included both typical learners and students at risk of developing reading problems. 

7. Two of the five participating teachers taught students who were repeating Kindergarten in “transition” classes and three were in general classrooms. 
Additionally, two self-contained classes of Kindergarteners with mild disabilities also participated in the study. Because appropriate controls did not 
exist for children in the self-contained classes, these classes were not in included in the review. 
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Research (continued) 



Effectiveness 



Fuchs et al. (2001) examined effects of Ladders to Literacy on 
Kindergarten students in an urban school district located in the 
Midwest. All students received their regular reading curriculum 
and the intervention group also received the Ladders to Literacy 
curriculum. Teachers were stratified by demographic and 
background features, and then randomly assigned to conditions. 
After teacher-level assignment, students of different ability levels 
were selected to be part of the study. The WWC focused on the 
portion of the study that included 11 teachers with 136 students 
in the intervention group and 11 teachers with135 students in the 
comparison group.® 



Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or moderate to large (see the What Works Clearinghouse 
Extent of Evidence Categorization Schemed The extent of 
evidence takes into account the number of studies and the 
total sample size across the studies that met WWC evidence 
standards with or without reservations.® 

The WWC considers the extent of evidence for Ladders to 
Literacy to be moderate to large for alphabetics and comprehen- 
sion and small for fluency. No studies that met WWC evidence 
standards with or without reservations addressed general read- 
ing achievement. 



Findings 

The WWC review of interventions for beginning reading 
addresses student outcomes in four domains: alphabetics, read- 
ing fluency, comprehension, and general reading achievement.''® 
The studies included in this report cover three domains: alpha- 
betics, fluency, and comprehension. Within alphabetics, results 
for four constructs are reported: phonemic awareness, phono- 
logical awareness, phonics, and letter knowledge. The findings 
below present the authors’ estimates and WWC-calculated 
estimates of the size and the statistical significance of the effects 
of Ladders to Literacy on students.'^ 



Aiphabetics. Four studies reviewed findings in the alphabetics 
domain. O’Connor (1999, Study A: Intensive Professional Devel- 
opment) found no statistically significant difference between the 
Ladders to Literacy group and comparison group performance 
on the phonemic awareness outcome (Test of Short-Term 
Memory), while the WWC found a statistically significant posi- 
tive effect.^® The study author found and the WWC confirmed 
statistically significant positive effects on two phonological 
awareness measures (Segmentation and Blending) and one 
standardized measure of phonics (Woodcock-Johnson (WJ) 
Letter-Word Identification subtest). The author did not report 



8. The study also included an analysis of the effects of Ladders to Literacy combined with Peer Assisted Learning (PALS). Although this comparison met 
evidence standards with reservations, it was not considered in the intervention rating because it went beyond the standard delivery of the program. 
However, results are reported in Appendices A4.1-A4.3. 

9. The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept, external validity, such as the students’ demographics and the types of 
settings in which studies took place, are not taken into account for the categorization. 

10. For definitions of the domains, see the Beginning Reading Protocol . 

11. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme 
for the formulas the WWC used to calculate the statistical significance. In the case of Ladders to Literacy, corrections for clustering and multiple com- 
parisons were needed for all three O’Connor studies and corrections for multiple comparisons were needed for Fuchs et al. (2001). 

12. The author reported results for the total sample and at-risk children, and the WWC focused only on typical learners. 
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The WWC found Ladders 
to Literacy to have 
potentially positive 
effects on alphabetics 
and fluency and mixed 
effects on comprehension 
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statistically significant results for Rhyme production. For the 
Letter Knowledge measure of Rapid Letter Naming, the author 
found a statistically significant effect; in WWC calculations, the 
effect was not statistically significant. The average effect size 
across the six outcomes was statistically significant. 

O’Connor (1999, Study B: Traditional Professional Develop- 
ment) found no statistically significant difference between the 
Ladders to Literacy group and comparison group performance 
on the phonemic awareness outcome (Test of Short-Term 
Memory). The author reported statistically significant effects 
of Ladders to Literacy across the phonological skills outcomes 
(Segmentation, Rhyme Production, and Blending) and measure 
of phonics (WJ Word Identification). The WWC confirmed statisti- 
cally significant effects only for the segmentation measure. The 
average effect size across the six outcomes was statistically 
significant. 

O’Connor et al. (1996) reported statistically significant effects 
of Ladders to Literacy across all measures (Sound Repetition, 
Blending, First Sound, Segmenting, Rhyme Production, Rapid 
Letter Naming, and the WJ Word Identification subtest). The 
WWC confirmed statistically significant effects only for the 
segmenting measure. The average effect size across the seven 
outcomes was not statistically significant but was large enough 
to be considered substantively important (an effect size greater 
than 0.25) according to WWC criteria. 

Fuchs et al. (2001) found positive and statistically significant 
effects of Ladders to Literacy on two phonological awareness 
measures (Segmentation and Blending); WWC confirmed the 



effect for the blending task, while the effect for segmentation 
was not statistically significant.^^ The authors found no statisti- 
cally significant difference between the Ladders to Literacy 
group and comparison group performance on three phonics 
measures— Rapid Letter Sound and two subtests of the Wood- 
cock Reading Mastery Test: Word Attack and Word Identifica- 
tion. The average effect size across the three outcomes was not 
statistically significant. 

Fiuency. O’Connor et al. (1996) found a positive, but not sta- 
tistically significant effect of Ladders to Literacy on the fluency 
outcome (Test of Oral Reading Fluency). 

Comprehension. Three studies examined the effects of Lad- 
ders to Literacy in the vocabulary construct (O’Connor, 1999, 
Study A: Intensive Professional Development; O’Connor, 1999, 
Study B: Traditional Professional Development; and O’Connor 
et al., 1996). The studies’ authors did not find a statistically 
significant effect of Ladders to Literacy on the Peabody Picture 
Vocabulary Test for any study. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as: positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme) . 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technical 
Details of WWC-Conducted Computations) . The improvement 



index represents the difference between the percentile rank 
of the average student in the intervention condition versus 
the percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 
index is based entirely on the size of the effect, regardless of 
the statistical significance of the effect, the study design, or the 



13. The size of the effect was not reported here because student-level data were not available to the WWC. Please see A3.1 for a more detailed explanation. 
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The WWC found Ladders to 
Uteracy to have potentially 
positive effects on 
alphabetics and fluency 
and mixed effects on 
comprehension (continued) 
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analyses. The improvement index can take on values between 
-50 and +50, with positive numbers denoting resuits favorabie to 
the intervention group. 

The average improvement index for aiphabetics is +25 
percentile points across three studies, with a range of +8 to 
+47 percentiie points across findings. One additional study that 
showed positive effects in alphabetics was not inciuded in this 
average and range because the student-level improvement index 
could not be computed by the WWC. 

The improvement index for fluency is +26 percentiie points for 
the singie finding in one study. 



The average improvement index for comprehension is +9 
percentiie points across the three studies, with a range of +1 to 
+17 percentiie points across findings. 

Summary 

The WWC reviewed six studies on Ladders to Literacyi''^ Four of 
these studies met WWC standards with reservations; the others 
did not meet WWC evidence screens. Based on these studies, 
the WWC found potentially positive effects on aiphabetics and 
fiuency and mixed effects on comprehension. The evidence 
presented in this report may change as new research emerges. 
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14. A single-case design study was identified but is not inciuded in this review because the WWC does not yet have standards for reviewing singie-case 
design studies. 

15. The sample is not appropriate to this review: the parameters for this WWC review specified that students should be in grades K-3 during the time of the 
intervention; this study did not focus on the targeted grades. 
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For more information about specific studies and WWC caicuiations, piease see the WWC Ladders to Literacy 
Technicai Appendices . 



16. Confound: this study included Ladders to Literacy but combined it with other interventions so the analysis could not separate the effects of the interven- 
tion from other factors. 

17. The disposition is pending development of WWC evidence standards for single subject designs. 
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Appendix 



Appendix A1.1 


Study characteristics: O’Connor, 1999, Study A: intensive Professionai Deveiopment (quasi-experimentai design) 


Characteristic 


Description 


Study citation 


O'Connor, R. E. (1999). Teachers Learning Ladders to Literacy. Learning Disabilities Research & Practice, 14{A), 203-214. (Study A: Intensive Professionai Development) 


Participants 


Two schoois voiunteered to impiement the intervention and two schools were recruited with a financiai incentive (contributions to their schooi iibraries) to serve as controls. 

Ten classes of 8 to 22 students participated in the study. Six of the ten teachers taught in general Kindergarten ciasses that included at least three children with disabilities. 
Two teachers taught in transition Kindergartens composed of students repeating Kindergarten (with or without diagnosed disabilities). Two teachers taught in special education 
classrooms. All classes included at least two children with disabilities. The children were predominantly European-American and African-American. Students were divided by 
ability level and labeled as typical learners or at-risk learners.'' Pretest equivalence was not established tor at-risk learners so this subgroup (including the two special educa- 
tion classes) was excluded trom the review.^ Therefore, this intervention report focuses on the findings reported only for the typical learners. The analysis sample of typical 
learners included 64 students in the intervention group and 41 students in the comparison group across general education and transition classes. 


Setting 


The study took place in four schools in an urban district. 


Intervention 


In addition to their typical pre-reading instruction, children in intervention classes were given more than twenty activities from the Ladders to Literacy book, including sound 
isolation, first sounds, rhyming pictures, rhyming, onsets and rhymes with first letters, invented spelling, story grammar, and integrating spelling and reading. The district- 
sponsored pre-reading curriculum included reading and discussing Big Books, learning letters of the alphabet and common sounds, and practicing writing of letters. 


Comparison 


Children in comparison classes received the same district-sponsored pre-reading curriculum as intervention classes. Children were also introduced to the concept of rhyme. 
Activities requiring blending or segmenting (beyond the initial sound) were not observed in any of the classes. The students were matched to the intervention students on the 
Peabody Picture Vocabulary Test and demographic variables. 


Primary outcomes 
and measurement 


For both pretest and posttest, the author administered the Peabody Picture Vocabulary test, the test of Short Term Memory, the Woodcock-Johnson Test of Achievement: 
Letter-Word Identification subtest, and four tasks: Rhyme Production, Segmenting, Blending, and Rapid Letter Naming. The Dictation subtest of the Woodcock Johnson Tests 
of Achievement was also used in the study but is not included because it is outside the scope of this Beginning Reading review. (See Appendices A2.1-2.3 for more detailed 
descriptions of outcome measures.) 


Teacher training 


In this intensive model of professional development, 14 days of teacher training were spread across the school year. Teachers discussed implementation of program activities, 
solved issues with materials, and shared data on the progress of their students. Teachers modeled instruction and rehearsed upcoming activities. Researchers worked with 
teachers to determine appropriate timing of activities and often observed students directly to supply the rationale for the next set of activities. 



1. At-risk learners were defined as children with low skills (children with high-incidence disabilities or whose standard scores fell below 85 at PPVT pretest). 

2. The groups are deemed non-equivalent if the pretest standardized mean difference is 0.5 or larger. See the Beginning Reading Protocol for more information 
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Appendix A1.2 Study characteristics: O’Connor, 1999, Study B: Traditionai Professionai Deveiopment (quasi-experimentai design) 



Characteristic 


Description 


Study citation 


O'Connor, R. E. (1999). Teachers Learning Ladders to Literacy. Learning Disabilities Research & Practice, 14{A), 203-214. (Study B: Traditional Professionai Development) 


Participants 


All the Kindergarten teachers in a district participated and were assigned to an in-service or comparison condition by geographical location, with all eight comparison ciasses 
coming from the same schooi. Students in each ciassroom were pre-tested and divided by ability level into typical learners and at-risk learners. In the combined analysis 
sample that included both subgroups, 192 students from nine classrooms were in the treatment group and 126 students from eight classrooms were in the comparison 
group.^ The children were predominantly of European-American descent. Class sizes ranged from 18-28 students and all included at least one child with a disability. 


Setting 


The study took place in a large rural midwestern district. 


Intervention 


In addition to their typical pre-reading instruction, children in nine intervention classes were given more than twenty Ladders to /./teracy activities, including sound isolation, 
first sounds, rhyming pictures, rhyming, onsets and rhymes with first letters, invented spelling, story grammar, and integrating spelling and reading. The district-sponsored 
pre-reading curriculum included reading and discussing Big Books, learning letters of the alphabet and common sounds, and practicing writing of letters. 


Comparison 


Children in eight comparison classes received the same district-sponsored pre-reading curriculum as the intervention group and were introduced to the concept of rhyme. The 
eight comparison classes formed a Kindergarten center in one school. The teachers of these classes routinely planned their instruction together and shared materials. The 
students were matched on the Peabody Picture vocabulary Test and demographic variables. 


Primary outcomes 
and measurement 


For both pretest and posttest, the author administered the Peabody Picture Vocabulary test, the test of Short Term Memory, the Letter-Word Identification subtest of the 
Woodcock-Johnson Test of Achievement, and the Rhyme Production, Segmenting, Blending, and Rapid Letter Naming tests. The Dictation subtest of the Woodcock Johnson 
Tests of Achievement was also used in the study but is not included because it is outside the scope of this Beginning Reading review. (See Appendices A2.1-2.3 for more 
detailed descriptions of outcome measures.) 


Teacher training 


Teacher participants included general education Kindergarten teachers. Title 1 teachers and other supporting staff. In this traditional model of professional development, 
teacher training totaled three and a half days spaced across the school year. Teachers discussed implementation of activities, solved issues with materials, and shared data on 
the progress of their students. Training provided teachers with modeling and rehearsal of upcoming activities. 



1 . Findings separated for at-risk iearners and typical learners are presented in Appendices A4.1-4.2 



WWC Intervention Report Ladders to Literacy for Kindergarten Students August 13, 2007 





Appendix A1.3 Study characteristics: O’Connor, Notari-Syverson, & Vadasy, 1996 (quasi-experimentai design) 



Characteristic 


Description 


Study citation 


O'Connor, R., Notari-Syverson, A., SVadasy, P. F. (1996). The effect of kindergarten phonological intervention on the first grade reading and writing of children with mild dis- 
abilities. Paper presented at the meeting of the American Educational Research Association, New York, NY. (ERIC Document Reproduction Service No. ED394129) 


Participants 


Three general Kindergarten teachers and two transition teachers (of children repeating Kindergarten) agreed to participate.^ The two transition teachers were randomiy 
assigned to treatment or control conditions. The three regular classroom teachers were not randomly assigned to the treatment or comparison condition. Students were 
matched by type of classroom (general or repeating Kindergarteners). In the combined analysis 42 students from three classrooms were in the intervention group and 24 
students from two classrooms were in the comparison group.^ Intervention and comparison students had comparable performance on pretests. The ethnic distribution in the 
school district, reflected in the composition of three general classes, was 52% African-American, 46% Caucasian, and 2% other ethnicities. The distribution in the transition 
classrooms was 65% African-American, 25% Caucasian, and 10% other ethnicities. Transition classes for children who repeat Kindergarten had reduced class sizes (12 to 15 
students compared to 21 to 25 students in the regular Kindergartens). Longitudinal findings at the end of first grade are presented in Appendix A4.3. 


Setting 


The study took place in a large, urban school district. 


Intervention 


The intervention was a supplement to a normal pre-reading instruction. Children in three intervention classes were given twenty-five Ladders to /./teracy activities over the 
6-month intervention period. In the first two months activities stimulated word and syllable awareness. The third and fourth months focused on rhyming, first sound isolation, 
and onset-rime level blending and segmenting. Letters and sounds were added to phonological activities in the final two months, when children were shown how to use a 
letter sound to match pictures that start the same. At this point the auditory blending games became more sophisticated, separating each spoken phoneme. In the two general 
Kindergartens, teachers conducted these activities in short sessions (5 to 15 minutes long) with their whole group of 21 to 25 students. In the transition class, the teachers 
and assistants usually conducted activities in smaller groups of 3 to 6 students. 


Comparison 


Children in two comparison classes received the same district-wide pre-reading curriculum as the intervention group. They did not practice auditory blending, segmenting 
words beyond the first letter, or selecting letters to represent sounds. The comparison transition class included finger-point reading of Big Books in addition to the general 
curriculum. 


Primary outcomes 
and measurement 


Eor both pretest and posttest, the authors administered the Peabody Picture Vocabulary test, the Letter-Word Identification subtest of the Woodcock-Johnson Test of Achieve- 
menf, and also Rhyme Production, Segmenting, Blending, Eirst Sound, Sound Repetition, and Rapid Letter Naming tests. Eor the follow-up tests conducted at the end of first 
grade, the authors administered a segmentation measure and the Word Attack and Word Identification subtests of the Woodcock-Johnson Test of Achievement. The Dictation 
subtest of the Woodcock Johnson Tests of Achievement and Test of Written Spelling were also used in the study but are not included because they are outside the scope of 
this Beginning Reading review. (See Appendices A2.1-2.3 for more detailed descriptions of outcome measures.) 


Teacher training 


Teachers in the intervention condition received 10 in-service training sessions spaced over the school year. Sessions reviewed materials, discussed the conceptual basis for 
each activify, and offered practical suggestions for incorporating activities into the class routines and feedback on implemenfation of earlier activities. Bi-weekly visits from the 
study authors extended teacher training by addressing particular classroom concerns, modifying activities, and monitoring program implementation. 



1. Two self-contained classes of Kindergarteners with mild disabilities also participated in the study. Because appropriate controls did not exist for children in the self-contained classes, these 
classes were not in included in the review. 

2. The analyses excluded children with disabilities in the general classrooms. 
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Appendix A1.4 Study characteristics: Fuchs et ai., 2001 (randomized controi triai with randomization prohiems) 



Characteristic 


Description 


Study citation 


Fuchs, G., Fuchs, L S., Thompson, A., AI Otaiba, S., Yen, L., Yang, N. J., Braun, M., and O'Conner, R. E. (2001). Is reading important in reading-readiness programs? A 
randomized fieid trial with teachers as program implementers. Journal of Educational Psychology 93(2), 251-267. 


Participants 


Thirty-three teachers were stratified and then randomly assigned to three conditions: Ladders to Literacy, Ladders to Literacy plus Peer-Assisted Learning Strategies (PALS), 
and a comparison condition. Students were selected to participate based on the Rapid Letter Naming test and student names were presented to teachers for their review 
and adjustment.'' For rating purposes, the WWC focused on the 11 teachers with 136 students that were in the Ladders to /./feracy treatment group and 11 teachers with 
135 students that were in the comparison group. ^ Students in both conditions were also compared longitudinally and in terms of varying levels of reading performance (high, 
medium, and low)^ in fall of first grade.'' 


Setting 


The study took place in four Title 1 and four non-Title 1 schools in the Nashville public school system. 


Intervention 


Children in the intervention classes received their typical pre-reading instruction and were given 15 Ladders to Ltocy activities for a maximum of 45 minutes a week for 
twenty weeks. These activities included word and syllable awareness, rhyming, first sound isolation, onset-rhyme blending, sound segmentation, journal writing, “letter sound 
of the week,” “morning message,” nursery rhymes and poems, and shared storybook reading. Only three of the activities presented students with printed letters. 

Students in the Ladders to Literacy plus PALS classes participated in a 20-week phonological awareness training and beginning decoding curriculum. The R4/.S component, 
which was implemented for 16 weeks, required children to work in pairs with peers of their own ages. The PALS activities focused on the correct sounds of letters and required 
children to read aloud sight words, decodable words, and simple sentences. 


Comparison 


Students in the comparison classes received their regular whole-class reading instruction. Nearly two-thirds of teachers used the school district’s formally adopted text: the 
Flarcourt-Brace Treasury of Literature: First Street Coiiection for Kindergarten. A majority of the teachers used First Street's Big Books, and about half of the teachers reported 
using High Hat A majority of comparison teachers taught alphabet letter naming. 


Primary outcomes 
and measurement 


For both pretest and posttest, the authors administered two subtests of the Woodcock Reading Mastery test (Word Identification and Word Attack), a rapid letter sound test, 
and a segmentation task. At posttest, the authors also administered a blending task. All these tests were also used during follow-up testing which occurred in the fall of first 
grade. The Spelling subtest of the Wechsler Individual Achievement test was also used in the study but is not included because it is outside the scope of this Beginning Read- 
ing review. (See Appendix A2.1-2.3 for more detailed descriptions of outcome measures.) 


Teacher training 


Intervention teachers attended a full-day workshop that included discussion of phonological awareness tasks and description of the 15 Ladders to Ltocy activities that teach- 
ers were asked to implement. Ladders to Literacy anti PALSteachers attended an additional half-day workshop to prepare their students in PALS. 



1 . Although teachers were randomly assigned to a treatment or Intervention condition, teachers’ judgment played a role In selecting students for the analysis sample. The WWC could not verify 
that student selection for the analysis sample was unrelated to treatment status, so the study met WWC evidence standards with reservations. 

2. Because it involved a variation of the intervention, the Ladders to Literacy plus PALS (phonological awareness plus beginning decoding) portion of the study is not included in this report. 

3. Reading performance was designated by the results of the Rapid Letter Naming pretest and teacher review/adjustment. 

4. These results are presented in Appendices A4. 
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Appendix A2.1 Outcome measures in the aiphabetics domain by construct 



Outcome measure 


Description 


Phonemic Awareness 
Test of Short Term Memory 


This test measured children’s ability to reproduce phonemes that varied from two to four phonemes in length (as cited in O’Connor, 1999). 


Phonological Awareness 
Segmentation/segmenting 


Segmentation with a test, similar to the Yopp-Singer test, that asks students to deconstruct words into component sounds (e.g. to say the sounds in cat) (as cited in Fuchs et 
al., 2001, and O’Connor etal., 1996). 


Blending 


Blending was tested using 22 three-sound words (e.g., soap, food, mom). The number of words correctly blended from sounds info a word in one minute was recorded (as 
cited in Fuchs et al., 2001). Correct responses were awarded two points and the examiner provided the answer for missed words. Next, the examiner presented any missed 
items in onset-rime format (s — oap), and correct responses were awarded one point (as cited in O’Connor et al., 1996). 


Rhyme production 


This test measured students’ ability to rhyme one-syllable words (as cited in O’Connor, 1999; O’Connor et al., 1996). 


First sound 


Items were scored correct if the child provided only the first sound of the word (e.g., for “pill”, /p/or/puh/ was correct; /pi/ was not) (as cited in O’Connor et al., 1996). 


Sound repetition 


For this test, children repeated isolated phonemes separated with a half-second pause (“Say exactly what 1 say :/p/ /!/ /f/”). Items varied from two to four in length, with the 
number of correct items reported (as cited in O’Connor, Notari-Syverson, and Vadasy, 1996). 


Letter Knowledge 
Rapid letter naming 


This test determines the number of letters a child can name correctly in one minute (as cited in Fuchs et al., 2001; O’Connor et al., 1996). 


Phonics 

Woodcock Reading Mastery 
Test; Word Attack subtest 


The standardized test measures phonemic decoding skills by asking students to read pseudowords. Students are aware that the words are not real (as cited in Fuchs et al., 
2001; O’Connor et al., 1996). 


Woodcock Johnson Tests of 
Achievement; Letter-Word 
Identification subtest. 


In this subtest, the examiner showed children letters or words, and children said the letter name or read the word (as cited in O’Connor, 1999; O’Connor et al., 1996). 


Rapid letter sound 


The test assesses the number of letter-sounds a student can name correctly in one minute. Students are shown a sheet with lower case letters and asked to tell the sound 
each letter makes. It was adapted from the work of Levy and Lysynchuk (as cited in Fuchs et al., 2001). 



Appendix A2.2 Outcome measures in the fluency domain 



Outcome measure 


Description 


Test of Oral Reading Fluency 


The test measures words read correctly in one minute on a primer level passage (as cited in O’Connor et al., 1996). 
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Appendix A2.3 Outcome measures in the comprehension domain 



Outcome measure 


Description 


Peabody Picture 
Vocabulary Test-R 


This individually administered norm-referenced test measured receptive vocabulary. Children are shown four pictures and asked to point to the one described by the examiner 
(as cited in O’Connor, 1999; O'Connor et ai., 1996). 
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Appendix A3.1 



Summary of study findings inciuded in the rating for the aiphahetics domain hy construct^ 



Authors’ findings from the study 



Mean outcome 

(standard deviation^) WWC calculations 













Mean difference'* 












Sample size 


Ladders to 




{Ladders to 




Statistical 






Study 


(teachers/ 


Literacy 


Comparison 


Literacy - 




significance^ 


Improvement 


Outcome measure 


sample 


students) 


groups 


group 


comparison) 


Effect size^ 


(at a = 0.05) 


index^ 



Construct Phonemic awareness 

Short term memory 


O’Connor, 1999; Study A; Intensive Professional Development (quasi-experimental design)^ 

Kindergarten 8/105 11.69 10,23 1.46 1,03 

(typical learners) (1.34) (1,51) 


Statistically 

significant 


+35 




O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)^ 






Short term memory 


Kindergarten 


17/318 


9.95 


9.61 


0,34 


0.19 


ns 


+8 








(1.69) 


(1.93) 










Construct Phonotogicat awareness 


















O’Connor, 1999; Study A; Intensive Professional Development (quasi-experimental design)^ 






Rhyme produotion 


Kindergarten 


8/105 


9,87 


9.44 


0.43 


0,26 


ns 


+10 




(typical learners) 




(1.32) 


(2.01) 










Segmentation 


Kindergarten 


8/105 


22.71 


10.69 


12,02 


1,78 


Statistically 


+46 




(typical learners) 




(6.72) 


(6.71) 






significant 




Blending 


Kindergarten 


8/105 


16.24 


11.89 


4,35 


0,83 


Statistically 


+30 




(typical learners) 




(4,55) 


(6.14) 






significant 






O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)^ 






Rhyme produotion 


Kindergarten 


17/318 


5.18 


4,22 


0,96 


0.23 


ns 


+9 








(4.06) 


(4,13) 










Segmentation 


Kindergarten 


17/318 


20.63 


11.39 


9.24 


0,93 


Statistically 


+32 








(9.64) 


(10.22) 






significant 




Blending 


Kindergarten 


17/318 


13.07 


10.18 


2.89 


0.52 


ns 


+20 








(5.38) 


(5.78) 














O’Connor et al, 1996 (quasi-experimental design)^ 








Sound repetition 


Kindergarten 


5/66 


10.90 


10.00 


0.90 


0,66 


ns 


+24 








(1.20) 


(1.60) 











(continued) 
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Appendix A3.1 



Summary of study findings inciuded in the rating for the aiphahetics domain hy construct^ (continued) 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

groups group 


Mean difference'* 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index^ 



Blending 


Kindergarten 


5/66 


15.50 

( 5 . 30 ) 


11.20 

( 6 . 9 ) 


4.30 


0.72 


ns 


- h 26 


First sound 


Kindergarten 


5/66 


10.30 

( 1 . 50 ) 


8.90 

( 2 . 80 ) 


1.40 


0.67 


ns 


+25 


Segmenting 


Kindergarten 


5/66 


23.70 

( 7 . 90 ) 


9.50 

( 5 . 8 ) 


14.20 


1.94 


Statistically 

significant 


+47 


Rhyme production 


Kindergarten 


5/66 


9.60 

( 1 . 6 ) 


9.10 

( 2 . 40 ) 


0.50 


0.26 


ns 


+)0 


Construct Letter knowtedge 




















O’Connor, 1999; Study A; Intensive Professional Development (quasi-experimental design) 






Rapid letter naming 


Kindergarten 


8/105 


45.50 


36.32 


9.18 


0.70 


ns 


+26 




(typicai learners) 




( 11 . 77 ) 


( 14 . 95 ) 












O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design) 






Rapid letter naming 


Kindergarten 


17/318 


36.67 


31.51 


5.16 


0.29 


ns 


-Hll 








( 17 . 19 ) 


( 18 . 42 ) 














O’Connor et al, 1996 (quasi-experimental design) 








Rapid Letter naming 


Kindergarten 


5/66 


38.80 


35.70 


3.10 


0.22 


ns 


- h 9 








( 13 . 10 ) 


( 15 . 40 ) 










Construct Phonics 




















O’Connor, 1999; Study A; Intensive Professional Development (quasi-experimental design) 






WJ Letter-Word Identification 


Kindergarten 


8/105 


16.14 


13.00 


3.14 


1.09 


Statistically 


- h 36 




(typicai iearners) 




( 3 . 44 ) 


( 1 . 52 ) 






significant 






O’Connor, 1999; Study B: Traditional Professional Development (quasi-experimental design) 






WJ Letter-Word Identification 


Kindergarten 


17/318 


13.45 


12.08 


1.37 


0.43 


ns 


-Hi 7 








( 2 . 96 ) 


( 3 . 55 ) 











(continued) 
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Appendix A3.1 



Summary of study findings inciuded in the rating for the aiphahetics domain hy construct^ (continued) 



Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Authors’ findings from the study 

Mean outcome 
(standard deviation^) 




WWC calculations 




Ladders to 
Literacy 
groups 


Comparison 

group 


Mean difference’* 
{Ladders to 
Literacy - 
comparison) 


Effect size^ 


Statistical 
significance^ 
(at a = 0.05) 


Improvement 

index^ 








O’Connor et al, 1996 (quasi-experimental design) 








WJ Letter-Word Identification 


Kindergarten 


5/66 


99.30 


96.10 


3.20 


0.29 


ns 


-Hll 








(11.20) 


(10.30) 














Fuchs et al., 2001^ (randomized controlled trial with randomization problems) 






Rapid letter sound 


Kindergarten 


22/271 


16.99 


15.81 


1.18 


na*° 


ns 


na*° 








(nr) 


(nr) 










WJ Word Attack 


Kindergarten 


22/271 


3.32 


2.03 


1.29 


na** 


ns 


na** 








(nr) 


(nr) 










WJ Word Identification 


Kindergarten 


22/271 


7.12 


5.47 


1.65 


na*2 


ns 


na*^ 








(nr) 


(nr) 










Average^^ for alphabetics domain (O’Connor, 1999, Study A) 








0.95 


Statistically 


-f33 
















significant 




Average^^ for alphabetics domain (O’Connor, 1999; Study B) 








0.43 


Statistically 


-Hi 7 
















significant 




Average^^ for alphabetics domain (O’Connor et al., 1996) 








0.68 


ns 


-f25 


Average^^ for alphabetics domain (Fuchs et al., 2001) 








na 


ns 


na 


Domain average^^ for alphabetics 










0.69 


na 


-f25 



ns = not statistically significant 
nr = not reported 
na = not applicable 



1 . This appendix reports findings considered for the effectiveness rating and the average improvement index. Subgroup findings and findings from the alternative intervention group from the same studies are not included in these ratings, 
but are reported in Appendices A4.1-A4.3. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The Ladders to Uteracy group mean equals the comparison group mean plus the mean difference. The computation of the mean difference took into account the pretest difference between the study groups. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -i-50, with positive numbers denoting results favorable to the intervention group. (continued) 
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Appendix A3.1 



Summary of study findings inciuded in the rating for the aiphahetics domain hy construct^ (continued) 



8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formula the WWC used to calculate statistical significance. 

In the case of O' Connor (1999) and O’Connor (1996), corrections for clustering and multiple comparisons were needed so the significance levels may differ from those reported In the original study. 

In the case of Fuchs et al. (2001), a correction for multiple comparisons was needed so the significance levels may differ from those reported In the original study. 

9. Student-level data were not available to the WWC. The authors' analysis provides a conservative test of the Intervention’s effectiveness. The study contributed to the Intervention ratings through the statistical significance of findings. 

10. Student-level standard deviations were not available for this study. Cluster-level standard deviations were 8.39 for the Intervention group and 6.20 for the comparison group. Because the student-level effect size and Improvement Index 
could not be computed, the magnitude of the effect size was not considered for rating purposes. However, the statistical significance for this study Is comparable to other studies and Is Included In the Intervention rating. For further 
details, please see Technical Details of WWC-Conducted Computations . 

1 1 . Student-level standard deviations were not available for this study. Cluster-level standard deviations were 3.46 for the Intervention group and 2.54 for the comparison group. Because the student-level effect size and Improvement Index 
could not be computed, the magnitude of the effect size was not considered for rating purposes. However, the statistical significance for this study Is comparable to other studies and Is Included In the Intervention rating. For further 
details, please see Technical Details of WWC-Conducted Computations . 

12. Student-level standard deviations were not available for this study. Cluster-level standard deviations were 6.71 for the intervention group and 4.76 for the comparison group. Because the student-level effect size and improvement index 
could not be computed, the magnitude of the effect size was not considered for rating purposes. However, the statistical significance for this study is comparable to other studies and is included in the intervention rating. For further 
details, please see Technical Details of WWC-Conducted Computations 

1 3. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect size. 
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Appendix A3.2 Summary of study findings inciuded in the rating for the fluency domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

group group 


Mean difference^ 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size^ (at a = 0.05) 


Improvement 

index^ 









O’Connor et al, 1996 (quasi-experimental design)^ 








Fluency 


Kindergarten 


5/66 


66.70 42.50 24.20 

(35.5) (29.00) 


0.72 


ns 


-f26 


Domain average^ for fluency 








0.72 


ns 


-f26 



ns = not statistically significant 

1 . This appendix reports findings at the end of first grade considered for the effectiveness rating and the improvement index. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

4. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -i-50, with positive numbers denoting results favorable to the intervention group. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formula the WWC used to calculate statistical significance. In the case of 0’ Connor et al (1996), a correction for 
clustering was needed so the significance level may differ from those reported in the original study. 

8. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. 
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Appendix A3.3 Summary of study findings inciuded in the rating for the comprehension domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

groups group 


Mean difference'* 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index^ 



Construct Vocabutary Devetopment 


O’Connor, 1999; Study A; Intensive Professional Development (quasi-experimental design)® 






PPVT 


Kindergarten 
(typical learners) 


8/105 


111.04 104.79 6.25 

(13.74) (13.76) 


0.45 


ns 


-f17 




O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)® 






PPVT 


Kindergarten 
(typical learners) 


17/318 


100.60 100.39 0.21 

(15.36) (16.15) 


0.01 


ns 


-Hi 








O’Connor et al., 1996; (quasi-experimental design)® 








PPVT 


Kindergarten 
(children without 
disabilities) 


5/66 


103.80 100.20 3.60 

(14.10) (16.10) 


0.24 


ns 


-f9 


Domain average® for comprehension 






0.23 


ns 


-f9 



ns = not statistically significant 

1 . This appendix reports findings considered for the effectiveness rating and the improvement index. Subgroup findings from the same studies are not included in these ratings, but are reported in Appendix A4.2. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The Ladders to Literacy group mean equals the comparison group mean plus the mean difference. The computation of the mean difference took into account the pretest difference between the study groups. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -i-50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formula the WWC used to calculate statistical significance. 

In the case of the 0’ Connor (1999) and O’Connor (1996) studies, corrections for clustering were needed, so the significance levels may differ from those reported in the original study. 

9. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect size. 
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Appendix A4.1 Summary of subgroup findings for aiphabetics domain by construct^ 



Authors’ findings from the study 



Mean outcome 

(standard deviation^) WWC calculations 













Mean difference'* 












Sample size 


Ladders to 




{Ladders to 




Statistical 






Study 


(teachers/ 


Literacy 


Comparison 


Literacy - 




significance^ 


Improvement 


Outcome measure 


sample 


students) 


groups 


group 


comparison) 


Effect size^ 


(at a = 0.05) 


index^ 



Construct: Phonemic Awareness 

O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)^ 



Short term memory 


Kindergarten 
(at-risk learners®) 


17/95 


9.15 

(2.08) 


8.43 

(2.36) 


0.72 


0.33 


ns 


-Hi 3 


Short term memory 


Kindergarten 
(typical learners) 


17/223 


10.29 

(1.36) 


10.10 

(1.48) 


0.19 


0.13 


ns 


-h5 


Construct Phonotogicat A wareness 

O’Connor, 1999; Study B: Traditional Professional Development (quasi-experimental design)^ 

Rhyme production Kindergarten 17/95 5.58 3.86 1.72 0.39 

(at-risk learners) (4.40) (4.29) 


ns 


-Hi 5 


Segmentation 


Kindergarten 
(at-risk learners) 


17/95 


17.59 

(11.39) 


6.46 

(8.61) 


11.13 


1.06 


Statistically 

significant 


-h36 


Biending 


Kindergarten 
(at-risk learners) 


17/95 


8.89 

(5.16) 


7.46 

(4.78) 


1.43 


0.28 


ns 


-hII 


Rhyme production 


Kindergarten 
(typical learners) 


17/223 


6.89 

(3.91) 


6.26 

(4.07) 


0.63 


0.16 


ns 


-h6 


Segmentation 


Kindergarten 
(typical learners) 


17/223 


21.95 

(8.49) 


13.44 

(10.17) 


8.51 


0.92 


Statistically 

significant 


-h32 


Blending 


Kindergarten 
(typical learners) 


17/223 


14.88 

(4.39) 


11.31 

(5.81) 


3.57 


0.71 


Statistically 

significant 


-h26 






Fuchs et al., 2001 (randomized controlled trial with randomization problems)^ 








Segmentation 


Kindergarten 


22/nr*® 


11.96 


8.93 


3.03 


na 


ns 


na 




(low achievers) 




(nr/6.71)** 


(nr/7.67)*® 










Segmentation 


Kindergarten 


22/nr*° 


15.76 


7.99 


7.77 


na 


Statistically 


na 




(average achievers) 




(nr/6.73)*“ 


(nr/5.74)*® 






significant 





(continued) 
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Appendix A4.1 Summary of subgroup findings for aiphabetics domain by construct (continued) 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

groups group 


Mean difference'* 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index^ 



Segmentation 


Kindergarten 
(high achievers) 


22/nr10 


22.96 

(nr/6.93)*“ 


14.16 

(nr/8.56)*“ 


8.80 


na 


Statistically 

significant 


na 


Blending 


Kindergarten 
(low achievers) 


22/nr*“ 


6,40 

(nr/4.1 6)*o 


2.48 

(nr/2.85)*“ 


3.92 


na 


Statistically 

significant 


na 


Blending 


Kindergarten 
(average achievers) 


22/nr*“ 


7.91 

(nr/3.56)*“ 


3,52 

(nr/3.1 8)*“ 


4.39 


na 


Statistically 

significant 


na 


Blending 


Kindergarten 
(high achievers) 


22/nr*o 


12.02 

(nr/4,54)*“ 


6.50 

(nr/5,07)*“ 


5.52 


na 


Statistically 

significant 


na 


Construct Letter Knowtedge 




















O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)® 






Rapid letter naming 


Kindergarten 
(at-risk learners) 


17/95 


26.32 

(15,75) 


18.11 

(13.46) 


8.21 


0,55 


ns 


-h21 


Construct Phonics 




















O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)® 






WJ Letter-Word identification 


Kindergarten 
(at-risk learners) 


17/95 


11.94 

(3.06) 


9.78 

(3.18) 


2,16 


0.69 


Statistically 

significant 


-h25 


WJ Letter-Word Identification 


Kindergarten 
(typical learners) 


17/223 


14.10 

(2.68) 


13.04 

(3.25) 


1.06 


0,36 


ns 


-Hi 4 






Fuchs et al., 2001 (randomized controlled trial with randomization problems)® 








Rapid letter sound 


Kindergarten 
(low achievers) 


22/nr*« 


9.89 

(nr/6,77)*“ 


8,49 

(nr/5.72)*“ 


1.40 


na 


ns 


na 


Rapid letter sound 


Kindergarten 
(average achievers) 


22/nr*o 


9.98 

(nr/7.46)*o 


14,28 

(nr/7.89)*“ 


-4.30 


na 


ns 


na 


Rapid letter sound 


Kindergarten 
(high achievers) 


22/nr*« 


24.72 

(nr/9,23) 


24.65 

(nr/7.43)*“ 


0.07 


na 


ns 


na 


WJ Word Attack 


Kindergarten 
(low achievers) 


22/nr*° 


0.61 

(nr/0.98)*“ 


0.41 

(nr/1. 00)*“ 


0.20 


na 


ns 


na 
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Appendix A4.1 Summary of subgroup findings for aiphabetics domain by construct (continued) 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

groups group 


Mean difference^ 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index^ 



WJ Word Attack 


Kindergarten 
(average achievers) 


22/nrio 


2.47 

(nr/4.30)i“ 


1.38 

(nr/ 2 . 02 )io 


1.09 


na 


ns 


na 


WJ Word Attack 


Kindergarten 
(high achievers) 


22/nrio 


6.88 

(nr/6.49)io 


4.29 

(nr/5.1 6)10 


2.59 


na 


ns 


na 


WJ Word Identification 


Kindergarten 
(low achievers) 


22/nrio 


1.34 

(nr/1. 28)^“ 


1.00 

(nr/1. 34)10 


0.34 


na 


ns 


na 


WJ Word Identification 


Kindergarten 
(average achievers) 


22/nrio 


5.63 

(nr/8.47)io 


2.87 

(nr/4.59)io 


2.76 


na 


ns 


na 


WJ Word Identification 


Kindergarten 
(high achievers) 


22/nrio 


14.42 

(nr/1 2.24)10 


12.54 
(nr/1 0.01)10 


1.88 


na 


ns 


na 



ns = not statistically significant 
nr = not reported 
na = not applicable 

1. This appendix presents subgroup findings — at-risk learners and typical learners (O’Connor, 1999) and low, average, and high achievers (Fuchs et al., 2001) — for measures that fall in the aiphabetics domain. Total group scores were 
used for rating purposes and are presented in Appendix A3.1 . 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The Ladders to /./feracy group mean equals the comparison group mean plus the mean difference. The computation of the mean difference took into account the pretest difference between the study groups. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -t-50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. 

In the case of O’Connor (1999; B), corrections for clustering were needed, so the significance levels may differ from those reported in the original study. 

In the case of Fuchs et al. (2001), no corrections were needed. 

9. At-risk learners were defined as children with low skills (i.e., children with high-incidence disabilities or whose standard scores fell below 85 at PPVT pretest). 

10. The total number of students was 271 with approximately 40% low achievers, 30% average achievers, and 30 high achievers; however, the exact number in each group was not provided. 

1 1 . Student-level standard deviations were not available for this study. Thus, a student-level effect size and improvement index could not be computed. Cluster-level standard deviations are presented in the author’s findings columns. 
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Appendix A4.2 Summary of subgroup findings for comprehension domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 

Literacy Comparison 

groups group 


Mean difference’* 
{Ladders to 
Literacy - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index^ 



Construct: Vocabulary Development 

O’Connor, 1999; Study B; Traditional Professional Development (quasi-experimental design)^ 



PPVT 


Kindergarten 


17/95 


93.06 


90.66 


2.40 


0.15 


ns 


-f6 




(at-risk learners) 




(14.98) 


(16.90) 










PPVT 


Kindergarten 
(typical learners) 


17/223 


103.87 

(14.39) 


104.43 

(14.05) 


-0.56 


-0.04 


ns 


-2 



ns = not statistically significant 

1 . This appendix presents subgroup findings (at-risk learners and typical learners) for measures that fall in comprehension domain. Total group scores were used for rating purposes and are presented in Appendix A3. 2. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The Ladders to Literacy group mean equals the comparison group mean plus the mean difference. The computation of the mean difference took into account the pretest difference between the study groups. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -t-50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Con ducted Co mputations for the formulas the WWC used to calculate 
statistical significance. 

In the case of O'Connor (1999; B), corrections for clustering were needed, so the significance levels may differ from those reported in the original study. 
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Appendix A4.3 Summary of longitudinal findings for the alphabetics domain by construct^ 



Authors’ findings from the study 



Mean outcome 

(standard deviation^) WWC calculations 



Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


Ladders to 
Literacy 
groups 


Comparison 

group 


Mean difference'* 
{Ladders to 
Literacy - 
comparison) 


Effect size^ 


Statistical 
significance^ 
(at a = 0.05) 


Improvement 

index^ 




Construct Phonological A wareness 


O’Connor et al, 1996 (quasi-experimental design)- 


—End of first grade^ 








Segmentation 


Kindergarten 


5/66 


31.40 

(12.40) 


27.50 

(8.50) 


3.90 


0.35 


ns 


-Hi 4 




Fuchs et al., 2001 (randomized controlled trial with randomization problems) — Fall of first grade^ 






Segmentation 


Kindergarten 
(low achievers) 


22/nr» 


21.23 
(nr/6.1 2)^ 


16.38 
(nr/7.1 1)^ 


4.85 


na 


ns 


na 


Segmentation 


Kindergarten 
(average achievers) 


22/nr8 


23.79 

(nr/8.49)^ 


18.42 

(nr/6.92)^ 


5.37 


na 


ns 


na 


Segmentation 


Kindergarten 
(high achievers) 


22/nr» 


25.64 

(nr/8.82)^ 


23.47 

(nr/7.70)^ 


2.17 


na 


ns 


na 


Segmentation 


Kindergarten 
(across categories) 


22/271 


23.56 

(nr/6.32)^ 


19.42 

(nr/4.55)^ 


4.14 


na 


ns 


na 


Blending 


Kindergarten 
(low achievers) 


22/nr» 


14.14 

(nr/2.66)^ 


9.22 

(nr/3.60)^ 


4.92 


na 


Statisticaiiy 

significant 


na 


Blending 


Kindergarten 
(average achievers) 


22/nr8 


14.47 

(nr/5.06)^ 


12.47 
(nr/3. 37)^ 


2.00 


na 


ns 


na 


Blending 


Kindergarten 
(high achievers) 


22/nr8 


16.78 

(nr/4.33)^ 


14.21 

(nr/4.95)^ 


2.57 


na 


ns 


na 


Blending 


Kindergarten 
(across categories) 


22/271 


15.13 

(nr/3.29)^ 


11.96 
(nr/3.1 8)^ 


3.17 


na 


Statistically 

significant 


na 


Construct Phonics 




O’Connor et al, 1996 (quasi-experimental design)- 


—End of first grade^ 








WJ Letter-Word identification 


Kindergarten 


5/66 


111.40 

(14.80) 


109.8 

(17.00) 


1.60 


0.10 


ns 


- h 4 


WJ Word attack 


Kindergarten 


5/66 


118.40 

(13.00) 


108.9 

(14.30) 


9.50 


0.70 


ns 


- h 26 


(continued) 
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Appendix A4.3 Summary of longitudinal findings for the alphabetics domain by construct^ (continued) 



Authors’ findings from the study 
Mean outcome 



Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


(standard deviation^) 




WWC calculations 




Ladders to 
Literacy 
groups 


Comparison 

group 


Mean difference'* 
{Ladders to 
Literacy - 
comparison) 


Effect size^ 


Statistical 
significance^ 
(at a = 0.05) 


Improvement 

index^ 






Fuchs et al., 2001 (randomized controlled trial with randomization problems) — Fall of first grade^ 






Rapid letter sound 


Kindergarten 


22/nr» 


21.25 


18.81 


2.44 


na 


ns 


na 




(low achievers) 




(nr/6,1 1)^ 


(nr/7.26)^ 










Rapid letter sound 


Kindergarten 


22/nr» 


29.18 


22.33 


6.85 


na 


ns 


na 




(average achievers) 




(nr/9,87)^ 


(nr/7.91 )^ 










Rapid letter sound 


Kindergarten 


22/nr» 


30.53 


34,24 


-3.71 


na 


ns 


na 




(high achievers) 




(nr/1 0.05)^ 


(nr/7.81)^ 










Rapid letter sound 


Kindergarten 


22/271 


26.98 


25,13 


1.85 


na 


ns 


na 




(across categories) 




(nr/6.39)^ 


(nr/6.1 2)^ 










WJ Word Attack 


Kindergarten 


22/nd 


3,46 


1.94 


1.52 


na 


ns 


na 




(low achievers) 




(nr/2.32)^ 


(nr/2.1 4)^ 










WJ Word Attack 


Kindergarten 


22/nd 


5.56 


3.03 


2.53 


na 


ns 


na 




(average achievers) 




(nr/5.1 3)^ 


(nr/2.1 4)^ 










WJ Word Attack 


Kindergarten 


22/nd 


8.85 


8.47 


0.38 


na 


ns 


na 




(high achievers) 




(nr/7,61)^ 


(nr/9.1 4)^ 










WJ Word Attack 


Kindergarten 


22/271 


5.96 


4.48 


1.48 


na 


ns 


na 




(across categories) 




(nr/4.70)^ 


(nr/4.31)^ 










WJ Word Identification 


Kindergarten 


22/nd 


10.20 


8.21 


1.99 


na 


ns 


na 




(low achievers) 




(nr/6.00)^ 


(nr/5,66)^ 










WJ Word Identification 


Kindergarten 


22/nd 


15.96 


13.95 


2.01 


na 


ns 


na 




(average achievers) 




(nr/8.09)^ 


(nr/7.45)^ 










WJ Word Identification 


Kindergarten 


22/nd 


26.57 


28.04 


-1.47 


na 


ns 


na 




(high achievers) 




CO 

CO 


(nr/1 3.54)^ 










WJ Word Identification 


Kindergarten 


22/271 


17.57 


16,73 


0,84 


na 


ns 


na 




(across categories) 




(nr/8.61)^ 


(nr/6,76)^ 










PPVT 


Kindergarten 


17/95 


93.06 


90.66 


2.40 


0.15 


ns 


-h6 




(at-risk iearners) 




(14.98) 


(16,90) 











(continued) 
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Appendix A4.3 Summary of longitudinal findings for the alphabetics domain by construct^ (continued) 



Outcome measure 







Authors’ findings from the study 

Mean outcome 
(standard deviation^) 




WWC calculations 




Study 


Sample size 
(teachers/ 


Ladders to 

Literacy Comparison 


Mean difference'* 
{Ladders to 
Literacy- 


Statistical 

significance^ 


Improvement 



sample 



students) 



group'’ 



group 



comparison) 



Effect size^ 



(at a = 0.05) 



index^ 



PPVT 


Kindergarten 


17/223 


103.87 


104.43 


-0.56 


-0.04 


ns 


-2 




(typical learners) 




(14.39) 


(14.05) 











ns = not statistically significant 

nr = not reported 

na = not applicable 

1. This appendix presents Ladders fo/./feracy group follow-up (first grade) data on measures that fall in the alphabetics domain. Findings that were used for rating purposes are presented in Appendix A3.1. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values be- 
tween -50 and -i-50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings 
not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate 
statistical significance. In the case of O'Connor et al. (1996), corrections for clustering were needed. In case of Fuchs et al. (2001), no corrections were needed. 

7. Student-level standard deviations were not available for this study. Thus, a student-level effect size and improvement index could not be computed, and are not presented in this table. Cluster-level standard deviations are presented in 

the author’s findings columns. 

8. The total number of students was 271 with approximately 40% low achievers, 30% average achievers, and 30 high achievers; however, the exact number in each group was not provided. 
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Appendix A5.1 Ladders to Uteracy rating for the aiphabetics domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentiaily positive, mixed, no discernibie effects, potentialiy negative, or negative.'' 

For the outcome domain of aiphabetics, the WWC rated Ladders to Literacy as having potentiaiiy positive effects. It did not meet the criteria for positive effects 
because there were no studies that met WWC evidence standards for a strong design. The remaining ratings (mixed effects, no discernibie effects, potentially negative 
effects, negative effects) were not considered, as Ladders to Literacy was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1; At least one study showing a statistically significant or substantively important positive effect. 

Met. Three studies showed statistically significant positive effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. There were no studies showing negative effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. There were no studies that met WWC standards for a strong design. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. There were no studies showing negative effects. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 Ladders to Uteracy rating for the fluency domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentiaily positive, mixed, no discernibie effects, potentialiy negative, or negative.'' 

For the outcome domain of fluency, the WWC rated Ladders to Literacy as having potentially positive effects. It did not meet the criteria for positive effects because 
there were no studies that met WWC evidence standards for a strong design. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, 
negative effects) were not considered, as Ladders to Literacy was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1; At least one study showing a statistically significant or substantively important positive effect. 

Met. One study showed substantively important positive effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. There were no studies showing negative effects and one study showed indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. There were no studies showing statistically significant positive effects, and no studies that met WWC evidence standards for a strong 
design. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. There were no studies showing negative effects. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.3 Ladders to Literacy rating for the comprehension domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentiaily positive, mixed, no discernibie effects, potentialiy negative, or negative.'' 

For the outcome domain of comprehension, the WWC rated Ladders to Literacy as having mixed effects. It did not meet the criteria for positive effects because no 
studies showed statistically significant positive effects. In addition, it did not meet the criteria for potentialiy positive effects because two studies showed indeterminate 
effects and only one study showed a substantively important positive effect. The remaining ratings (no discernible effects, potentially negative effects, and negative 
effects) were not considered as Ladders to Literacy was assigned the highest applicable rating. 

Rating received 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. One study of Ladders to Literacy had a substantively important positive effect in this domain. No studies showed a statistically significant 
or substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing a 
statistically significant or substantively important effect. 

Met. One study of Ladders to Literacy had a substantively important positive effect in this domain and two studies showed indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No study of Ladders to Literacy had a statistically significant positive effect in this domain. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies of Ladders to Literacy showed statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. One study of Ladders to Literacy had a substantively important positive effect in this domain. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important pos/f/ve effects. 

Not met. No studies of Ladders to Literacy showed statistically significant or substantively important negative effects in this domain, and more 
studies showed indeterminate effects (two) than substantively important positive effects (one) in this domain. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A6 Extent of evidence rating by domain 



Outcome domain 


Number of studies 


Schoois 


Sample size 

Students 


Extent of evidence^ 


Alphabetics 


4 


over 14 


760 


Moderate to large 


Fluency 


1 


over 1 


66 


Small 


Comprehension 


3 


over 6 


489 


Moderate to large 


General reading achievement 


0 


0 


0 


na 



na = not applicable/not studied 

1 . A rating of “moderate to large” requires at least two studies and two schools across studies in one domain, and a total sample size across studies of at least 350 students or 14 classrooms. 
Otherwise, the rating is “small.” 
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